Pandas Profiling Report

Dataset statistics

Number of variables	10
Number of observations	824
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	64.5 KiB
Average record size in memory	80.2 B

Variable types

Numeric	10

Warnings

`water` is highly correlated with `superplasticizer`	High correlation
`superplasticizer` is highly correlated with `water`	High correlation
`water` is highly correlated with `superplasticizer`	High correlation
`superplasticizer` is highly correlated with `water`	High correlation
`age` is highly correlated with `csMPa`	High correlation
`csMPa` is highly correlated with `age`	High correlation
`water` is highly correlated with `superplasticizer`	High correlation
`superplasticizer` is highly correlated with `water`	High correlation
`csMPa` is highly correlated with `Id` and 2 other fields	High correlation
`fineaggregate` is highly correlated with `Id` and 5 other fields	High correlation
`Id` is highly correlated with `csMPa` and 7 other fields	High correlation
`water` is highly correlated with `fineaggregate` and 5 other fields	High correlation
`coarseaggregate` is highly correlated with `fineaggregate` and 6 other fields	High correlation
`slag` is highly correlated with `fineaggregate` and 5 other fields	High correlation
`cement` is highly correlated with `csMPa` and 7 other fields	High correlation
`flyash` is highly correlated with `Id` and 3 other fields	High correlation
`superplasticizer` is highly correlated with `csMPa` and 7 other fields	High correlation
`Id` has unique values	Unique
`slag` has 377 (45.8%) zeros	Zeros
`flyash` has 461 (55.9%) zeros	Zeros
`superplasticizer` has 304 (36.9%) zeros	Zeros

Reproduction

Analysis started	2021-11-23 13:19:51.511416
Analysis finished	2021-11-23 13:20:04.659527
Duration	13.15 seconds
Software version	pandas-profiling v3.0.0
Download configuration	config.json

Id
Real number (ℝ_≥0)

HIGH CORRELATION
UNIQUE

Distinct	824
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	513.8470874

Minimum	0
Maximum	1028
Zeros	1
Zeros (%)	0.1%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	0
5-th percentile	50.15
Q1	251.75
median	513.5
Q3	770.25
95-th percentile	974.85
Maximum	1028
Range	1028
Interquartile range (IQR)	518.5

Descriptive statistics

Standard deviation	296.7867789
Coefficient of variation (CV)	0.5775780115
Kurtosis	-1.20657835
Mean	513.8470874
Median Absolute Deviation (MAD)	259.5
Skewness	0.000627465727
Sum	423410
Variance	88082.39214
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	1	0.1%
693	1	0.1%
678	1	0.1%
680	1	0.1%
681	1	0.1%
683	1	0.1%
684	1	0.1%
686	1	0.1%
688	1	0.1%
690	1	0.1%
Other values (814)	814	98.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	0.1%
1	1	0.1%
4	1	0.1%
5	1	0.1%
6	1	0.1%
7	1	0.1%
8	1	0.1%
9	1	0.1%
11	1	0.1%
12	1	0.1%

Value	Count	Frequency (%)
1028	1	0.1%
1027	1	0.1%
1026	1	0.1%
1024	1	0.1%
1023	1	0.1%
1022	1	0.1%
1021	1	0.1%
1019	1	0.1%
1018	1	0.1%
1017	1	0.1%

cement
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	254
Distinct (%)	30.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	283.360801

Minimum	102
Maximum	540
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	102
5-th percentile	143.615
Q1	192
median	275.1
Q3	359.9
95-th percentile	491
Maximum	540
Range	438
Interquartile range (IQR)	167.9

Descriptive statistics

Standard deviation	107.5364039
Coefficient of variation (CV)	0.3795034581
Kurtosis	-0.6077758768
Mean	283.360801
Median Absolute Deviation (MAD)	83.9
Skewness	0.4933427896
Sum	233489.3
Variance	11564.07816
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
425	17	2.1%
362.6	16	1.9%
475	13	1.6%
251.4	13	1.6%
310	13	1.6%
250	12	1.5%
349	12	1.5%
446	11	1.3%
236	10	1.2%
331	10	1.2%
Other values (244)	697	84.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
102	4	0.5%
108.3	4	0.5%
116	3	0.4%
122.6	4	0.5%
132	2	0.2%
133	3	0.4%
133.1	1	0.1%
134.7	1	0.1%
135	1	0.1%
135.7	2	0.2%

Value	Count	Frequency (%)
540	7	0.8%
531.3	5	0.6%
528	1	0.1%
525	7	0.8%
522	2	0.2%
520	2	0.2%
516	2	0.2%
500.1	1	0.1%
500	10	1.2%
491	7	0.8%

slag
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	166
Distinct (%)	20.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	74.37160194

Minimum	0
Maximum	359.4
Zeros	377
Zeros (%)	45.8%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	22
Q3	144.775
95-th percentile	236
Maximum	359.4
Range	359.4
Interquartile range (IQR)	144.775

Descriptive statistics

Standard deviation	86.97778445
Coefficient of variation (CV)	1.169502635
Kurtosis	-0.5182968517
Mean	74.37160194
Median Absolute Deviation (MAD)	22
Skewness	0.8020652798
Sum	61282.2
Variance	7565.134988
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	377	45.8%
189	24	2.9%
106.3	17	2.1%
24	11	1.3%
20	9	1.1%
98.1	9	1.1%
19	8	1.0%
145	8	1.0%
26	7	0.8%
116	6	0.7%
Other values (156)	348	42.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	377	45.8%
11	4	0.5%
13.6	2	0.2%
15	5	0.6%
17.2	1	0.1%
17.5	1	0.1%
17.6	1	0.1%
19	8	1.0%
20	9	1.1%
22	6	0.7%

Value	Count	Frequency (%)
359.4	2	0.2%
342.1	1	0.1%
316.1	2	0.2%
305.3	3	0.4%
290.2	2	0.2%
288	4	0.5%
282.8	3	0.4%
272.8	1	0.1%
262.2	5	0.6%
260	1	0.1%

flyash
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	130
Distinct (%)	15.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	53.16080097

Minimum	0
Maximum	195
Zeros	461
Zeros (%)	55.9%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	118.3
95-th percentile	166.85
Maximum	195
Range	195
Interquartile range (IQR)	118.3

Descriptive statistics

Standard deviation	64.0006463
Coefficient of variation (CV)	1.203906734
Kurtosis	-1.321913729
Mean	53.16080097
Median Absolute Deviation (MAD)	0
Skewness	0.5660377685
Sum	43804.5
Variance	4096.082726
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	461	55.9%
118.3	15	1.8%
141	14	1.7%
24.5	13	1.6%
79	11	1.3%
121.6	9	1.1%
94	9	1.1%
95.7	9	1.1%
100.4	8	1.0%
167	8	1.0%
Other values (120)	267	32.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	461	55.9%
24.5	13	1.6%
59	1	0.1%
71	1	0.1%
71.5	1	0.1%
76	1	0.1%
77	2	0.2%
78	2	0.2%
78.4	1	0.1%
79	11	1.3%

Value	Count	Frequency (%)
195	3	0.4%
194.9	1	0.1%
194	1	0.1%
190	1	0.1%
185.3	1	0.1%
185	2	0.2%
184	1	0.1%
183.9	1	0.1%
182.1	1	0.1%
182	1	0.1%

water
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	179
Distinct (%)	21.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	181.7970874

Minimum	121.8
Maximum	247
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	121.8
5-th percentile	146.13
Q1	164.9
median	185.35
Q3	192
95-th percentile	228
Maximum	247
Range	125.2
Interquartile range (IQR)	27.1

Descriptive statistics

Standard deviation	21.32190452
Coefficient of variation (CV)	0.1172840821
Kurtosis	0.1765762128
Mean	181.7970874
Median Absolute Deviation (MAD)	13
Skewness	0.09197296622
Sum	149800.8
Variance	454.6236124
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
192	97	11.8%
228	43	5.2%
185.7	36	4.4%
203.5	30	3.6%
186	25	3.0%
162	17	2.1%
164.9	16	1.9%
153.5	13	1.6%
200	12	1.5%
193	11	1.3%
Other values (169)	524	63.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
121.8	5	0.6%
126.6	4	0.5%
137.8	3	0.4%
140	1	0.1%
140.8	4	0.5%
141.8	5	0.6%
142	1	0.1%
143.3	2	0.2%
144.7	4	0.5%
145	3	0.4%

Value	Count	Frequency (%)
247	1	0.1%
246.9	1	0.1%
237	1	0.1%
236.7	1	0.1%
228	43	5.2%
221.4	1	0.1%
221	1	0.1%
220.1	1	0.1%
220	2	0.2%
219.7	1	0.1%

superplasticizer
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	105
Distinct (%)	12.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	6.163956311

Minimum	0
Maximum	32.2
Zeros	304
Zeros (%)	36.9%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	6.1
Q3	10.125
95-th percentile	16.085
Maximum	32.2
Range	32.2
Interquartile range (IQR)	10.125

Descriptive statistics

Standard deviation	5.967257716
Coefficient of variation (CV)	0.9680889051
Kurtosis	1.265712251
Mean	6.163956311
Median Absolute Deviation (MAD)	5.6
Skewness	0.8977497366
Sum	5079.1
Variance	35.60816464
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	304	36.9%
11.6	28	3.4%
8	17	2.1%
7	14	1.7%
9.9	13	1.6%
7.8	13	1.6%
16.5	13	1.6%
9	13	1.6%
6	12	1.5%
11	12	1.5%
Other values (95)	385	46.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	304	36.9%
1.7	4	0.5%
1.9	1	0.1%
2	1	0.1%
2.5	2	0.2%
3	6	0.7%
3.1	1	0.1%
3.4	3	0.4%
3.6	5	0.6%
3.9	7	0.8%

Value	Count	Frequency (%)
32.2	3	0.4%
28.2	5	0.6%
23.4	4	0.5%
22.1	1	0.1%
22	5	0.6%
20.8	1	0.1%
19	1	0.1%
18.6	4	0.5%
18.3	1	0.1%
18	1	0.1%

coarseaggregate
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	258
Distinct (%)	31.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	973.5485437

Minimum	801
Maximum	1145
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	801
5-th percentile	842
Q1	932
median	968
Q3	1040.6
95-th percentile	1104.51
Maximum	1145
Range	344
Interquartile range (IQR)	108.6

Descriptive statistics

Standard deviation	78.69463012
Coefficient of variation (CV)	0.08083277473
Kurtosis	-0.6438676258
Mean	973.5485437
Median Absolute Deviation (MAD)	52.5
Skewness	-0.04148492161
Sum	802204
Variance	6192.84481
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
932	46	5.6%
852.1	39	4.7%
944.7	24	2.9%
1125	21	2.5%
968	20	2.4%
967	16	1.9%
1047	15	1.8%
942	10	1.2%
822	9	1.1%
938	9	1.1%
Other values (248)	615	74.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
801	4	0.5%
801.4	1	0.1%
811	1	0.1%
814	1	0.1%
814.1	1	0.1%
817.9	1	0.1%
819	1	0.1%
819.2	1	0.1%
820	1	0.1%
822	9	1.1%

Value	Count	Frequency (%)
1145	1	0.1%
1134.3	5	0.6%
1130	1	0.1%
1125	21	2.5%
1124.4	2	0.2%
1120	1	0.1%
1119	2	0.2%
1118.8	1	0.1%
1118	1	0.1%
1113	1	0.1%

fineaggregate
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	274
Distinct (%)	33.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	772.1074029

Minimum	594
Maximum	992.6
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	594
5-th percentile	613
Q1	726.775
median	778.5
Q3	821.25
95-th percentile	895.895
Maximum	992.6
Range	398.6
Interquartile range (IQR)	94.475

Descriptive statistics

Standard deviation	80.98471665
Coefficient of variation (CV)	0.1048878904
Kurtosis	-0.1344581102
Mean	772.1074029
Median Absolute Deviation (MAD)	45.5
Skewness	-0.2399879142
Sum	636216.5
Variance	6558.524332
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
755.8	24	2.9%
594	24	2.9%
613	20	2.4%
670	18	2.2%
801	14	1.7%
746.6	13	1.6%
887.1	13	1.6%
712	11	1.3%
845	10	1.2%
780.1	10	1.2%
Other values (264)	667	80.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
594	24	2.9%
605	5	0.6%
611.8	5	0.6%
612	1	0.1%
613	20	2.4%
613.2	2	0.2%
623	2	0.2%
630	3	0.4%
631	4	0.5%
633	2	0.2%

Value	Count	Frequency (%)
992.6	4	0.5%
945	2	0.2%
943.1	4	0.5%
942	4	0.5%
925.7	4	0.5%
905.9	4	0.5%
903.8	3	0.4%
903.6	4	0.5%
901.8	4	0.5%
900.9	4	0.5%

age
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	14
Distinct (%)	1.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	44.66140777

Minimum	1
Maximum	365
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	1
5-th percentile	3
Q1	12.25
median	28
Q3	56
95-th percentile	180
Maximum	365
Range	364
Interquartile range (IQR)	43.75

Descriptive statistics

Standard deviation	60.47570164
Coefficient of variation (CV)	1.354093045
Kurtosis	13.07549467
Mean	44.66140777
Median Absolute Deviation (MAD)	21
Skewness	3.33541121
Sum	36801
Variance	3657.310489
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=14)

Value	Count	Frequency (%)
28	350	42.5%
3	110	13.3%
7	94	11.4%
56	72	8.7%
14	46	5.6%
90	44	5.3%
100	39	4.7%
180	21	2.5%
91	20	2.4%
270	9	1.1%
Other values (4)	19	2.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	2	0.2%
3	110	13.3%
7	94	11.4%
14	46	5.6%
28	350	42.5%
56	72	8.7%
90	44	5.3%
91	20	2.4%
100	39	4.7%
120	3	0.4%

Value	Count	Frequency (%)
365	9	1.1%
360	5	0.6%
270	9	1.1%
180	21	2.5%
120	3	0.4%
100	39	4.7%
91	20	2.4%
90	44	5.3%
56	72	8.7%
28	350	42.5%

csMPa
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION

Distinct	701
Distinct (%)	85.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	35.85786408

Minimum	2.33
Maximum	82.6
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	6.6 KiB

Quantile statistics

Minimum	2.33
5-th percentile	11.3645
Q1	23.685
median	34.08
Q3	45.8625
95-th percentile	67.28
Maximum	82.6
Range	80.27
Interquartile range (IQR)	22.1775

Descriptive statistics

Standard deviation	16.86509934
Coefficient of variation (CV)	0.4703319557
Kurtosis	-0.2738606121
Mean	35.85786408
Median Absolute Deviation (MAD)	10.81
Skewness	0.4619332841
Sum	29546.88
Variance	284.4315757
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
33.4	4	0.5%
23.52	4	0.5%
77.3	4	0.5%
71.3	4	0.5%
79.3	4	0.5%
39.3	3	0.4%
31.35	3	0.4%
43.7	3	0.4%
64.3	3	0.4%
17.54	3	0.4%
Other values (691)	789	95.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2.33	1	0.1%
3.32	1	0.1%
4.57	1	0.1%
4.78	1	0.1%
4.9	1	0.1%
6.27	1	0.1%
6.47	1	0.1%
6.81	1	0.1%
6.94	1	0.1%
7.32	1	0.1%

Value	Count	Frequency (%)
82.6	1	0.1%
81.75	1	0.1%
80.2	1	0.1%
79.99	1	0.1%
79.4	1	0.1%
79.3	4	0.5%
78.8	1	0.1%
77.3	4	0.5%
76.8	1	0.1%
76.24	1	0.1%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows

	Id	cement	slag	flyash	water	superplasticizer	coarseaggregate	fineaggregate	age	csMPa
0	995	158.6	148.9	116.0	175.1	15.0	953.3	719.7	28	27.68
1	507	424.0	22.0	132.0	178.0	8.5	822.0	750.0	28	62.05
2	334	275.1	0.0	121.4	159.5	9.9	1053.6	777.5	3	23.80
3	848	252.0	97.0	76.0	194.0	8.0	835.0	821.0	28	33.40
4	294	168.9	42.2	124.3	158.3	10.8	1080.8	796.2	3	7.40
5	286	181.4	0.0	167.0	169.6	7.6	1055.6	777.8	28	27.77
6	938	154.8	183.4	0.0	193.3	9.1	1047.4	696.7	28	18.29
7	447	178.0	129.8	118.6	179.9	3.6	1007.3	746.8	56	48.59
8	692	212.0	141.3	0.0	203.5	0.0	973.4	750.0	90	39.70
9	652	102.0	153.0	0.0	192.0	0.0	887.0	942.0	3	4.57

Last rows

	Id	cement	slag	flyash	water	superplasticizer	coarseaggregate	fineaggregate	age	csMPa
814	308	277.1	0.0	97.4	160.6	11.8	973.9	875.6	100	55.64
815	661	141.3	212.0	0.0	203.5	0.0	971.8	748.5	7	10.39
816	130	323.7	282.8	0.0	183.8	10.3	942.7	659.9	28	74.70
817	663	133.0	200.0	0.0	192.0	0.0	927.4	839.2	28	27.87
818	871	159.0	187.0	0.0	176.0	11.0	990.0	789.0	28	32.76
819	87	286.3	200.9	0.0	144.7	11.2	1004.6	803.7	3	24.40
820	330	246.8	0.0	125.1	143.3	12.0	1086.8	800.9	14	42.22
821	466	190.3	0.0	125.2	166.6	9.9	1079.0	798.9	100	33.56
822	121	475.0	118.8	0.0	181.1	8.9	852.1	781.5	28	68.30
823	860	314.0	0.0	113.0	170.0	10.0	925.0	783.0	28	38.46

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows